Imaging method and system

ABSTRACT

There is provided a method for compositing remote sensing images (such as satellite images in visible light) with user generated content. An image capture device carried on a satellite platform is tasked to capture at least one remote sensing image of a geographical area within a predetermined time slot. The at least one remote sensing image is received at an interface portal along with at least one user-generated content, UGC, item. UGC items that have an associated location within the geographical area and that were captured within the time slot are merged, by the interface portal, with the remote sensing image to form a hybrid image. The interface portal and user device implementing the above method are described as is the system in which the method is implemented.

TECHNICAL FIELD

The present disclosure relates to a method and system for generatingcomposite images.

BACKGROUND

The ubiquity of mobile telephones and their increasing connectivity is,at least in part, tied to the mass adoption of social media services bytheir users. Mobile telephones having built-in camera functions (and inparticular smartphones) enable a social trend towardnear-instantaneously shared experience. Many social network servicesencourage the sharing of photographic images (still and video) capturedusing the mobile's camera(s).

Smartphones commonly have both a front-facing camera (i.e. facing theuser when the user looks at the phone's screen) and a rear-facingcamera, so that it has become very simple for the user to operate thecamera function to capture an image of themselves in a “selfie”. Theterm selfie is commonly used to refer to any photograph taken by, andincluding, the photographer; often the photographer intends tophotograph themselves alongside other people, and/or with animals,scenery, architecture, etc. More recently, certain mobile manufacturershave introduced enhanced selfie-like functions, such as the facility totake “bothies”—a “bothie” combines the images captured by each of thesecameras in a single composite image).

Selfies allow the user to memorialise otherwise ephemeral real-timeexperiences, placing the user in the context of an event that may bepersonal, limited to the user's friendship group or more globallyshared.

Outdoor experiences, such as rock concerts, sporting events, nationaland religious events, even natural phenomena, often involve a great manyparticipants and movement across a geographic area. It can be difficultto depict the scale of such events and the context of the individualwithin them. The selfie allows a personal snapshot of the photographerat an event but may not adequately reflect where the individual is orhow that location relates to the fleeting moments that make up theevent. It is entirely possible to take a selfie at an event but to findthat the selfie does not begin to capture the excitement and uniquenessof being present at that event at that moment.

Reference to any prior art in this specification is not anacknowledgement or suggestion that this prior art forms part of thecommon general knowledge in any jurisdiction, or globally, or that thisprior art could reasonably be expected to be understood, regarded asrelevant/or combined with other pieces of prior art by a person skilledin the art.

It is an object of the invention to at least ameliorate one or more ofthe above or other shortcomings of prior art and/or to provide a usefulalternative.

SUMMARY OF THE INVENTION

The invention is a method and apparatus as defined in the appendedclaims.

It will be appreciated that features and aspects of the presentdisclosure may be combined with other different aspects of thedisclosure as appropriate, and not just in the specific illustrativecombinations described herein.

BRIEF DESCRIPTION OF DRAWINGS

The present disclosed subject matter will be understood and appreciatedmore fully from the following detailed description taken in conjunctionwith the drawings in which corresponding or like numerals or charactersindicate corresponding or like components. Unless indicated otherwise,the drawings provide exemplary embodiments or aspects of the disclosureand do not limit the scope of the disclosure. In the drawings:

FIG. 1 schematically illustrates the flow of operations in accordancewith an aspect of the present disclosure;

FIGS. 2A and 2B illustrate two different scenarios in which the methodof the present disclosure may be implemented;

FIG. 3 illustrates the operation of a mobile application executing inthe or each user device when obtaining a user generated content item inaccordance with an aspect of the present disclosure;

FIG. 4 schematically illustrates certain operations performed inaccordance with an aspect of the present disclosure;

FIG. 5 schematically illustrates an operation of polling for remotesensing images that are ready for use in certain implementations of thepresent disclosure;

FIG. 6 schematically illustrates certain operations performed whenprocessing user generated content items with remote sensing images inaccordance with an aspect of the present disclosure;

FIG. 7 schematically illustrates a case where additional content iscombined with hybrid images generated from user generated content itemsmerged with remote sensing images in accordance with an aspect of thepresent disclosure;

FIG. 8 illustrates a mechanism for managing the dissemination of thehybrid image content in accordance with an aspect of the presentdisclosure;

FIG. 9 depicts screen-views of a user interface presented by a mobileapplication in the implementation of an aspect of the presentdisclosure;

FIG. 10 illustrates a further screen-view of a user interface presentedby a mobile application in the implementation of an aspect of thepresent disclosure;

FIG. 11 illustrates yet another screen-view of a user interfacepresented by a mobile application in the implementation of an aspect ofthe present disclosure; and

FIG. 12 schematically illustrates certain functional elements of aninterface portal in accordance with a further aspect of the presentdisclosure.

DESCRIPTION OF EXAMPLE EMBODIMENTS

The detailed description set forth below in connection with the appendeddrawings is intended as a description of various exemplary embodimentsof the disclosure and is not intended to represent the only forms inwhich the present disclosure may be practiced. It is to be understoodthat the same or equivalent functions may be accomplished by differentembodiments that are intended to be encompassed within the scope of theinvention. Furthermore, terms such as “comprises”, “comprising”, “has”,“contains” or any other grammatical variation thereof, are intended tocover a non-exclusive inclusion, such that module, circuit, devicecomponents, structures and method steps that comprises a list ofelements or steps does not include only those elements but may includeother elements or steps not expressly listed or inherent to such module,circuit, device components or steps. An element or step proceeded by“comprises . . . a” does not, without more constraints, preclude theexistence of additional identical elements or steps that comprise theelement or step.

The present disclosure seeks to blend the personal context of anevent-goer captured in user generated content (such as a selfie, bothieor the like) with a relevant and inclusive overall image thatnevertheless relates to the shared experience of the event-goer at amoment in time and a precise location.

The user of the internet is increasingly familiar with the ease of“mash-ups” between library map data and geo-location information storedas metadata to media content such as still and video images obtained byimage capture devices. Furthermore, it has become conventional formobile devices such as smartphones and wearable devices (particularlysmartwatches) to incorporate a satellite positioning system unit (usingtechnologies such as GPS, GLONASS, and/or BeiDou, for example) capableof providing an accurate geo-location for the device. As a result, mediacontent (e.g. user generated content, UGC) generated by such devices mayhave associated metadata providing a geo-location of the device at thetime the content is captured.

Thus, where a user takes a selfie, the user's device may be configuredto append geographical coordinates to metadata of the selfie (along withother information such as a time stamp and/or camera settings). Theselfie could then be associated with a location in a map offered by amapping service, using the associated geographical coordinates. Themapping service might access a library image of the location and thesurrounding region at a predetermine resolution. The library image mightbe: schematic, showing features of interest to a traveler such as roads,rivers and places; geographic, showing natural features and (altitude)contours of the landscape; or aerial views, often themselves capturedfrom imaging platforms such as aircraft, unmanned aerial vehicles (UAV)or satellites.

Aerial view library images are typically obtained from historical data.Satellite images are available of the overwhelming majority of theEarth's surface but they are not typically renewed regularly: evenpopulated regions may be represented by satellite images that were takensome years previously. Recent changes to locations may evade capture insuch image libraries, for example the appearance and disappearance ofshort-duration phenomena such as the temporary camp surrounding anoutdoor music festival or snow cover in a territory that rarelyexperiences sub-zero temperatures. Furthermore, the images of an area insuch libraries may be assembled from images captured by differenttechniques, from different imaging platforms and at differentresolutions/exposures/illumination angles: this can often lead tolibrary imaging of a particular geographic region being inconsistent.Weather phenomena such as cloud formations can mean that the landsurface of some regions is not visible in the available library images.

In many events, the use of UAVs (also referred to as “drones”) isdiscouraged (or banned outright) on the grounds of safety forevent-goers. Likewise, the use of manned imaging platforms such ashelicopters can introduce an unnecessary risk as well as contributingunwelcome noise, and if they are sufficiently low-altitude, down-drafteffects.

While mobile phone users have become accustomed to near instantaneousimages being captured by their mobile devices, they are often unable tocombine these snapshot images (whether still or video) with a meaningfulremote image that represents their experience.

In the case of a user attending an outdoor music festival in a muddyfield in rural England, the ability of a mapping service to provide animage of that field, captured some years ago on a sunny spring day, willresult in a disappointing and inaccurate representation of the livedexperience.

Satellite platforms for remote sensing and imaging equipment are oftenarranged to orbit the Earth so that they pass directly over differentregions of the Earth's surface (i.e. they are not usually“geostationary”). It is possible to arrange for an imaging device borneby a satellite to target a geographical strip at a particular time(capturing one or more images of that strip), where the strip of thesurface is a region of the Earth within a predetermined acute viewingangle from the normal substantially under the path of the satellite. Theinstructions for scheduling a targeted satellite activity are instancesof a more general category of instructions between ground systems andthe operational functions of the satellite known as “tasking” thesatellite; however, unless the context requires otherwise, the term“tasking” used herein refers to image capture scheduling instructions.

By providing a facility in which an event organiser or sponsor may taskthe capture of remote sensing images of a predetermined region in aprearranged timeslot, and a facility allowing one or more users toobtain their own respective UGC at a time within the timeslot while theyare located at a location within the predetermined region, the user (ormore likely a plurality of users) may (each) be provided with asatellite snapshot of the geographic region where the UGC is capturedthat represents the overall aerial view of the region at the time theUGC was captured. Participating users may be identified throughregistration with a satellite-selfie service and authenticated use of acorresponding service application (e.g. an “app”) on their device basedon their registration.

Thus, while attending the mud-affected English music festival, the usermay be instructed (by their app) to be ready for a “satellite selfie”image to be taken by the satellite overhead, in a scheduled timeslot,for the five minutes between 12:30 to 12:35 on the Saturday of the musicfestival, say. The user is encouraged to take a selfie of herself with agroup of her fellow concert-goers during the scheduled timeslot—theselfie including a GPS location reading from the user's mobile phone atthe time the selfie was taken. The GPS location may then be used toidentify the user's accurate position in an image of the muddy, festivalfield as it was when the selfie was taken (i.e. allowing the selfieimage taken during the event/venue to be synchronised with satellitephotographs of the event/venue).

Satellite tasking has typically been considered the sole responsibilityof the satellite owner in the past due to the complexity and proprietarynature of the operation of a satellite system, the sensitivity of thesatellite equipment and the monetary cost of the associated maintenance,monitoring and scheduling tasks. As a result, there has been norequirement for third party satellite tasking.

In certain embodiments, the app may notify a user (registered with asatellite-selfie service) that a suitable satellite is overhead and theuser himself (through his device) may provide the instruction taskingthe satellite to capture satellite imagery: effectively requesting asatellite-equivalent of a selfie. In this case, the location informationfor the user's device and the current time may be used to calculatewhether the satellite is due to be in a position where remote imagecapture of the user's location is feasible. As above, a satellite imageis captured of the user's location (i.e. the location of the locationunit associated with the user's device) at the time of the transmissionof the user instruction during an event that has provisioned orsubscribed to the service.

Whether the user or the event organiser tasks the satellite imagecapture, the user can be notified of the window in time during the eventwhen the service is available (i.e. the satellite is overhead) and canperform a content generation action. Where the time of the satelliteimage capture is set by the event organiser (when the organiser istasking the satellite), the content generation action may include theaction of taking a selfie during the window of time and determining anassociated geo-location.

The content generation action may be automated by the app on the deviceto occur at a time pre-set by the user (so that the user knows theselfie will be taken at the same predetermined time the satellite imagecapture is tasked to occur) or by an event controller transmitting asignal from event systems. Alternatively, the user could manually sendthe request e.g. by using the app to take a conventional selfie, thataction then acting as the tasking trigger for the satellite (orpseudo-satellite) image capture. In the latter case, there may be asignificant time difference between the time of the conventional selfieand the time at which the tasked satellite is able to respond to thetasked instruction, due to latency in the processing of the taskingtrigger.

Where an event controller issues a content generation trigger signal,that is used to instruct the user device (e.g. to take a selfie) and totask the satellite, it can be arranged that this signal is transmittedto different user devices at slightly different times: staggering thetrigger transmission ensures that the satellite's tasking system is notoverloaded.

One benefit of the method of the present disclosure, in this example, isthat it facilitates a far truer depiction of the user's experience of anevent.

FIG. 1 schematically illustrates the flow of operations in accordancewith an aspect of the present disclosure.

In functional block 102, at least one remote sensing image of apredetermined geographical area is received from an image capture devicecarried on a satellite platform. The image capture device having beentasked to capture the at least one remote sensing image within apredetermined time slot.

In functional block 104, a user-generated content, UGC, item, isobtained. The UGC item has an associated location within thegeographical area and will have been captured at a user-determined timewithin the time slot. In certain cases, the UGC item further includes astill image, such as a selfie.

It is noted that the operations in functional blocks 102 and 104 may beperformed in either order, or they may indeed be performed substantiallysimultaneously.

Once both UGC item and remote sensing image become available, theoperations of functional block 106 are performed, merging the UGC itemand the remote sensing image to form a hybrid image.

The hybrid image associated with a short-duration event can then beshared, through social networking and other communication services: thehybrid image has both social and evidential value (e.g. the user hastangible evidence that they were really there at a given event, as seenin the selfie or other UGC item in amongst a crowd of attendees, thatcan be seen in the associated satellite image). Examples of UGC itemsinclude video content, sound recordings, heart rate traces, biometricreadings, etc., from the user. Optionally, further content and metadatamay be associated with the hybrid image, for example additionalmultimodal media content (such as “official” video footage, concertprogramme listings, additional environment/weather readings, commercialoffers, shared UGC from other users, etc.) and processing effects (suchas a zoom-in effect or the addition of filters or “skins” to the hybridimage).

FIGS. 2A and 2B illustrate two different scenarios in which the methodof the present disclosure may be implemented.

FIG. 2A shows a satellite 202 (i.e. a remote imaging platform) having acommunication link to an interface portal 204 (via an antenna 206, aground station 208 and a communications network 210). The interfaceportal 204 is also in communication with a user device 220 (here, asmartphone or a tablet computer incorporating a media content unit and ageo-location unit). Images captured by the media content unit of theuser device are stored with location information determined at the timeof the image capture. Provided the image capture time and location areeach with a range of predetermined values associated with an event, anapplication executing on the user device may transmit the captured mediacontent to the interface portal via the communications network. Thetransmission of captured media content need not be instantaneous: in thecase of a tablet computer, for example, the content may be captured buttransmission may be postponed until the tablet is in range of a suitablecommunication connection (for a tablet computer lacking cellularcommunication functionality, this may mean waiting until a publiclyaccessible WiFi connection is available before transmission ispossible).

FIG. 2B also shows a satellite 202 having a communication link to aninterface portal 204 (via an antenna 206, a ground station 208 and acommunications network 210). The interface portal 204 is also incommunication with a user device 224. The user device 224 is incommunicative contact with an external media content unit 222 (here, anevent camera). Either the event camera or the user device may becommunicatively coupled to a geo-location unit (not shown), from whichlocation information is obtained.

The external media content unit 222 may advantageously be portable orprovided on a non-fixed platform, e.g. mounted on a rail mount, a cablemount or a UAV/drone. In such cases, the external media content unit 222may conveniently be collocated with a geo-location unit.

Alternatively, the external media content unit 222 may have a knownfixed location (as would be the case for cameras located in a sportsstadium). Images captured by the media content unit 224 are stored withlocation information determined at the time of the image capture. Anapplication executing on the user device 224 may instruct thetransmission of the captured media content from the media content unit222 to the interface portal 204 via the communications network 210,provided the image capture time and location are each with a range ofpredetermined values associated with an event.

Facial recognition may be implemented so that the external media contentunit 222 may capture an image of the user in the crowd, identify him andcapture a personalised image. Alternatively, the external media contentunit 222 may be instructed to capture an image of a fixed location (forexample, a meeting point or performance area location or a predeterminedseat, in each case the location having a fixed position relative to theexternal media content unit 222 and thus a unique imaging angle. In eachcase, the user device 224 triggers the image capture through thecommunicative contact with the external media content unit 222.

In a first embodiment of the present disclosure, the system that allowsthe method of the present disclosure to be implemented includes a mediacontent unit for obtaining media content, a unit for providing ageo-location to be associated with the media content (this may beintegral in the media content device), a user device (typically a mobiletelephone) executing a mobile application, a remote imaging platform, aportal for interfacing with the or each user, as well as the remoteimaging platform and with at least one administrator.

Examples of media content unit include cameras (for both still and videocapture), audio recording devices, heart rate monitors, and biometricsensors.

Examples of geo-location units include satellite positioning systemunits (using technologies such as GPS, GLONASS, and/or BeiDou, forexample) and cellular communication units (capable of locating a devicewithin a cellular network using conventional cell ID techniques).

In certain embodiments the media content unit and/or geo-location unitare provided within the user device. The mobile application provides auser interface to the interface portal.

The remote imaging platform is advantageously a satellite or apseudo-satellite. An example of a suitable pseudo-satellite is a deviceknown as a stratospheric UAV, which may remain aloft over apredetermined geographic area for extended periods (e.g. of the order ofweeks). Such pseudo-satellite platforms are typically operated ataltitudes significantly higher than commercial air transport, whereoperations are not considered a security or safety risk to users on theground. In the following description, reference to remote imagingplatforms and satellites includes conventional geo-stationary andorbital satellites as well as high altitude pseudo-satellites.

The operations illustrated in FIG. 1 are performed by the interfaceportal. The portal includes at least one processor, which executesinstructions to implement the method of the present disclosure. Theinterface portal may advantageously be implemented in one or more thanone networked servers. The instructions may include instructions for aplatform application programming interface (API), a management portaland a processing application.

The platform API is arranged to perform a number of services. It mustfacilitate the exchange of data between database and the varioussystems/devices (including the management portal, processing applicationand associated application queues, product delivery databases, themobile application and other external services such as externalsatellite tasking, delivery systems and file storage (for uploads andassets), and the content database).

In particular, the platform API may be configured to manage the useraccounts for the (or each) user of the system. The user may thenregister a new account, update a corresponding user profile for thataccount, as well as login and logout from that account to sign up toparticipate in any given satellite image opportunity (or group of suchopportunities) associated with an event they are attending.

The platform API also provides access to an events database, detailingthe location and timing of the events and whether the user is permittedaccess to this information. Event records in the events database may beconfigured as “invitation only” or “upon request” for example: so thataccess to the event information may be restricted to particular users(or groups of users) at the behest of an administrator. The platform APImay also facilitate the handling of those event signups and/or theissuing of invitations to one or more users.

Media content obtained by the media content unit may be received by theinterface portal via a communication facility (this facility may handlethe upload and download of selfie images and other user generatedcontent, for example, from user devices to the interface portal).

The platform API is also provided with a facility for integration withthe operations of the remote imaging platform. The remote sensing imagescaptured by the remote imaging platform are received by the interfaceportal via the communication facility.

In certain embodiments, the platform API may be responsible forretrieval of additional content such as video embed code for sharing andfor the retrieval of user generated content from social networkingservices, from the user's own historical data and/or from the historicaldata of other users that has been shared with the user.

Media content may not be in a format suitable for immediate sharing. Theplatform API is provided with a facility for passing media content thathas been obtained from user devices to the processing application forprocessing. The platform API also monitors the status of the processingof this content so that the user (and administrators) may check thestatus.

The processing application may provide services including a schedulerservice, queuing and processing services, data storage integration,messaging (including mobile and web-based push notifications), imageprocessing/generation, and video processing. The processing applicationmay be standalone and scalable to a plurality of servers within theinterface portal. The processing application may include a plurality ofschedulers including the upload poller and may additionally readinstructions from an associated application queue, and carry out imageand video processing tasks. The processing application mayadvantageously be implemented as logically separate, dedicated stillimage and video processing applications.

In certain embodiments, the user experience is mediated by themanagement portal executing on the interface portal. Through themanagement portal, an administrator may manage content related to eventsincluding metadata, text, video, images which will be delivered via theplatform API and may allow advertisers and event organisers to provideupdated information, including event scheduling and commercialofferings, for dissemination to users. The management portal includes afacility for managing login from the administrators, event organisersand advertisers (collectively referred to as partners) and for managingthe records in the events database (allowing their creation, updates andtransmission of invitations). The administrator may be given “superuser” rights, so that new partners can be added to the platform andtheir accounts can be managed.

In certain embodiments, the management portal serves as an interfacewith the satellite service allowing the tracking of the satellite, theperformance of an availability check. This interface includes a facilityfor satellite tasking, necessary for coordinating the remote imaging ofan event with the actions of one or more users attending that event.

In the user device, the mobile application includes features for userregistration and login and for viewing information about past andupcoming events for which the enhanced selfie experience is available.

The mobile application maintains a list of upcoming events and thecorresponding details of those events.

Through the application, the user may register to join (i.e.participate) in one or more such events, whether by responding to anevent invitation from the portal API or as a user-originating request.

Where the user has participated in previous events, the application maystore details of those events and associated hybrid images.

In certain embodiments, the application generates active event updatesand may present a countdown so that the user is made aware of theimminence of the next slot at which user generate content generation canbe coordinated with a pass by a suitably tasked remote sensing platform(i.e. satellite).

In certain embodiments, the application facilitates communication withan external media content unit (i.e. an image capture device, forexample the camera of an official photographer or of an event photobooth): the application, then, may be provided with a user interfacethat allows the user to trigger image capture in the external imagecapture device and/or to request an image obtained by such a device. Inother embodiments, the application is logically coupled to an in-builtcamera function of the user device and facilitates a selfie capability.

In certain embodiments, the geo-location unit is external to the userdevice. In other embodiments, the user device incorporates thegeo-location unit. In each case, the mobile application is arranged toaccess the location information generated by the geo-location unit andto use this information in the provision of location-based features,such as event mapping, location meta-tagging and ‘geo-fencing’.

The mobile application may be arranged to receive push notificationsrelated to the event and the status of processing of hybrid images atthe interface portal.

The mobile application may be provided with a social networking serviceinterface so that hybrid images can be uploaded and shared to (as wellas downloaded to) third-party services such as Facebook®, Instagram®,Twitter®, etc.

In certain embodiments, the hybrid image may be extended by inclusion ofdata from at least one additional remote sensing image. This may allowthe user to pinpoint his location in more than one image of the event,even to select a preferred image (as he might wish if a cloud or kiteobstructed the view for one of the remote sensing images). While theadditional remote sensing image may be captured from the same satelliteplatform, even the same image capture device, certain embodiments havethe remote sensing image captured from a further image capture device onthe same satellite platform (e.g. an infrared image taken alongside theconventional visible light image) and other embodiments have the remotesensing image captured from an image capture device on a differentplatform entirely. That different platform may be a second satelliteplatform; a UAV platform; a manned aircraft platform; or even an eventcamera on a gantry.

In certain embodiments, the hybrid image may be extended by inclusion ofdata from a historical source. Thus, historical aerial and satelliteimages of the geographical area may be obtained as historical remotesensing images. The inclusion of historical remote sensing images alongwith the near-real-time hybrid images means that the resulting compoundmultimedia item can depict not only the experience of being at an eventbut also the ephemeral nature of the event.

Where the event relates to an organised beach-cleaning event, forexample, the resulting compound multimedia item may illustrate theresults of the even-participants activities through a sequence ofbefore-during-afterwards images while also placing the user within theevent.

The various steps in the method illustrated in FIG. 1 are set out inmore detail in FIGS. 3-8, described below.

In FIG. 3, the operation of the mobile application executing in the oreach user device is illustrated. At step 302, the user device eithercaptures a user generated content item (or obtains a content itemcaptured by an external media content unit upon instruction from theuser). A location of the user device (or the media content unit) isdetermined at step 304. In certain embodiments, location is determinedby an external geo-location unit; in other cases the geo-location unitis integral to the user device.

The user device then queues the UGC item (with its associated locationinformation) for upload to the interface portal, in an applicationqueue. Alternatively, the UGC item may be queued for upload directlyfrom the media content unit. In certain embodiments, the user device ormedia content unit may operate to upload the UGC item to a queue in theinterface portal itself or in a dedicated queue proxy communicativelycoupled to the interface portal.

FIG. 4 sets out the operations at the remote imaging platform and in theinterface portal in more detail.

One or more remote sensing images are captured (by an imaging devicecarried by a satellite) of a predetermined region of the Earth's surfaceat a time coinciding with an event, step 402.

The or each remote sensing image is transmitted from the satellite (viaintermediate ground station and network nodes) to the interface where itis received, step 404.

The image is then processed for use in the enhancement of the UGC items,step 406. The image may be cropped to cover a specific geographicalarea, and depending upon the angle from which the satellite viewed thetarget area (compared to the vertical, when the satellite is directlyover the target area) the remote sensing image may also beorthorectified (i.e. processed into a map projection so that the spatialcoordinates accurately overlap the coordinates in conventional 2D maps).

Optionally, the interface portal (through the processing application)may apply routines to test the resulting image and to verify that thesatellite image fulfils usability requirements, step 408 (this mayinclude applying a score to each image of a series of similar images ofthe same event/time). Where a plurality of images are available andcloud cover is determined to affect some images in a first region of theimage and other images in a second region, different from the firstregion, this may mean that only images unaffected by cloud cover in theregion including a predetermined location (such as the reported locationof a user) will be verified for use.

Once processing is complete, the resulting image (or images) are markedas ready, step 410. The marked images may be stored in the interfaceportal for future use.

In certain embodiments, as illustrated in FIG. 5, the interface portalexecutes a “generation poller” routine of the processing application atscheduled intervals, step 502. The poller routine periodicallydetermines whether one or more remote sensing (e.g. satellite) image ismarked as ready, step 504. Where no satellite image is marked ready, theprocessing application returns to a monitoring state until the nextpolling occurs. In the case there is a remote sensing image ready, thescheduling process within, or external to, the processing applicationthen accesses an application queue of UGC items and receives the or eachUGC item held in the respective queue, step 506. Each UGC item is thenprocessed along with a (ready) remote sensing image, step 508.

A more detailed illustration of the process of FIG. 1, is shown in FIG.6. FIG. 6 also details the operations performed in the interface portalat step 508 of FIG. 5.

The processing of the images is performed in by the processingapplication executing on the interface portal. At step 604, theprocessing application obtains a remote sensing image for an event(which may be the satellite image marked as ready and discovered at step504). A UGC item is also obtained, step 606 (the UGC item havingcorresponding location information, in metadata for example). In FIG. 6,it is assumed that the UGC item is associated with the event shown inthe remote sensing image: this may be because the UGC item includes atimestamp, location information and/or a user registration thatcorrespond to the time range, remote-imaged region and/or register forthat event.

The location associated with the UGC item is converted into a pixellocation within the remote sensing image, step 608. Thus, the interfaceportal calculates the coordinates for the point in the remote-imagedregion that corresponds to the geo-location where the UGC item wascaptured. For instance, where the UGC item is a selfie, the GPS locationof the device taking the selfie when the selfie is taken is pinpointedon a satellite image of the region containing that GPS location.

An indicator, such as a flag, pin or target icon, is then imposed on theremote sensing image, step 610, so that the remote sensing image isdisplayed with the location from the UGC item visibly marked.

The image data from the remote sensing image is then composited with thedata of the UGC item to generate a hybrid image, step 612. Whendisplayed, the hybrid image is arranged to show at least a portion ofthe UGC item and at least a portion of the remote sensing image, markedto show the location from the UGC item.

Optionally, the processing application may construct a compoundmultimedia item by composing additional content with the hybrid image,step 614.

The construction of compound multimedia item may include insertingmetadata from the event organiser and/or from a third party (such as acharity or sponsor or ticketing service). In certain examples, theadditional content is an additional video sequence (such as aprofessional recording of a live performance of the band on stage at thetime of the hybrid image). In certain examples, the additional contentincludes prepared text and/or graphical content (such as artwork) thatmay be used in a template for a compound multimedia item (converting thehybrid image into a personalised official record of event participation,for instance).

FIG. 7 illustrates a case where the optional additional content of step614 include static video (for example a video “sting” applied to allhybrid images for a particular event or a promotional content item froman event sponsor or organiser) and provides a dynamic zoom effect.Static video content is received from a static content store (which maybe local to the interface portal or external), step 702. Other examplesof additional content include personalised messages, music, hyperlinksto further content and services and themes (i.e. backdrops, filters,etc.).

The remote sensing image may be viewed at a range of differentmagnifications. A dynamic zoom effect may be created by generating aplurality of views of a location depicted in the remote sensing image atsuccessively high magnification and concatenating the views in order ofincreasing magnification as frames of a video sequence, step 704. Thefinal frame may be made to correspond to the portion of the remote imagecontaining the marked location now forming part of the hybrid image. Thelocation of the UGC item may be marked in some or all of the precedingframes in the dynamic zoom sequence. For example, a video effectdepicting a “zoom” from a view of the satellite image at relatively lowmagnification to a view of the portion of the satellite imagesurrounding the marked location at a predetermined high magnificationmay be prepended to a hybrid satellite and selfie image of an event sothat the location of the selfie is given context in the lowmagnification (i.e. geographically wider) satellite view of the event.

The static video content, dynamic zoom effect “video” and hybrid imagemay then be assembled in a compound multimedia item, step 706. Thecompound multimedia item is then stored, step 708, and if necessaryuploaded to a server, step 710, e.g. a video sharing service server fromwhich registered users may share their video images for the event.

It is envisaged that, during any one event, there may one or more timeslots at which a remote sensing image may be tasked to capture one ormore remote sensing images. Furthermore, for mass-participation events,such as music festivals, sports events or national events, there will bea great number of UGC items captured by a similar number of differentregistered users. As the number of items and increases, it can beexpected that the time taken to process each of these UGC items intohybrid images (or compound multimedia items) will increase. Assuming theprocessing and storage facilities are not infinite, this potential lagbetween UGC capture and delivery of a hybrid image ready for sharingneeds to be managed.

FIG. 8 illustrates one mechanism for managing the dissemination of thehybrid image content. A scheduling application polls the processingapplication at scheduled intervals (e.g. every n minutes, where n iswhole number) to determine whether any given UGC item (or batch of suchitems) has been processed into one or more hybrid items, step 802. Ifone or more hybrid image determined to be ready to share (at step 804),a message is sent to the mobile application in the user device, step 806(for example a push notification including a unique reference assignedto the hybrid image, whereby the hybrid image may be identified). Whenthe mobile application receives the message, step 808, the user maydownload the hybrid image from the interface portal and view and/orshare that hybrid image through the mobile application and/or anassociated website.

The user interface presented by the mobile application in a display ofthe user device is illustrated in FIG. 9. In a first screen-view 920 a,the application shows an example of a captured UGC item 902 (e.g. aselfie image) and associated location information 906 (representing thegeographical location of the user device at the time the UGC item wascaptured).

In a second screen-view 920 b, a portion of a remote sensing image 904is displayed. The location information 906 is represented as a pin 910at a corresponding point in the remote sensing image 904.

FIG. 10 illustrates an example screen-view 1020 of a hybrid imagegenerated from remote sensing image 904 marked with pin 910 at alocation corresponding to the location 906 at which UGC item 902 wascaptured and overlaid with an inset scaled-down image 1002 correspondingto the UGC item 902. In the illustrated example, an overlaid userinteraction box 1006 is displayed, showing various additional functions(represented as icons) the mobile application may offer such as thepossibility to download the image or to attach it to a message such asan email. The mobile application may integrate with a hybrid imagesharing service, in which case a sharing option 1008 may be provided tofacilitate sharing across the sharing service.

When the user wishes to recall an event, a record of the hybrid imagefrom that event may form part of a compound multimedia item stored inthe user device and accessible through the mobile application. FIG. 11illustrates a screen-view 1120 displaying an item 1106 (relating to anevent called YourFest) and a further item 1108 (relating to anotherhybrid image related to the same event or a different event). Here thedisplayed items may be displayed showing reduced scale versions, or“thumbnails”, of the constituent UGC item and remote sensing image. Thedisplayed item may include a overlaid user interaction box 1102presenting various additional functions (represented as icons) that theuser may trigger without opening the compound multimedia item in themobile application.

The interface portal 204 of FIGS. 2A and 2B may be implemented as aserver 1200, as shown in FIG. 12. The interface portal may beimplemented as a distributed network of such servers.

The interface portal 1200 includes: a processing unit 1210, which itselfincludes at least one processor (here, a plurality of processors 1212,1214); a memory unit 1230; input/output components 1250; and a bus 1202,through which each of the components of the interface portal arecommunicatively connected.

The memory unit 1230 includes at least one memory device. FIG. 12 showsthe memory unit 1230 with both a main memory 1232 and a storage unit1236. Examples of main memory include volatile RAM, whereas examples ofstorage unit include non-volatile hard disk and solid-state memorydevices. Instructions 1216 for the operation of the processing unit arestored in at least one of the volatile and non-volatile memory devicesof the memory unit.

The input/output components 1250 include at least one communicationunit: examples of communication unit include wired communication units1262, such as a network interface card, NIC, for wired ethernetconnectivity, and wireless communication elements 1264, such as2G/3G/4G/5G cellular and/or WiFi modules.

The interface portal 1200 may communicate with one or more user devices1270 via a direct communication link 1272 or indirectly 1282 via anetwork 1260, such as the Internet. The interface portal 1200 may alsocommunicate with an administration server 1280 (via the network 1260 andcommunication link 1274. Communication with other external systems 1290,including the ground station servers of a satellite operating company,social network and sharing service servers, advertisement servers,presence servers, static video servers, etc., is provided by respectivecommunication links 1276 via the network 1260.

The description of the various embodiments of the present disclosure hasbeen presented for purposes of illustration and example, but is notintended to be exhaustive or to limit the invention to the formsdisclosed. It will be appreciated by those skilled in the art thatchanges could be made to the embodiments described above withoutdeparting from the broad inventive concept thereof.

Further particular and preferred aspects of the present invention areset out in the accompanying independent and dependent claims. It will beappreciated that features of the dependent claims may be combined withfeatures of the independent claims in combinations other than thoseexplicitly set out in the claims.

1. A computer-implemented method comprising: receiving at least oneremote sensing image of a predetermined geographical area from an imagecapture device carried on a satellite platform, the image capture devicehaving been tasked to capture the at least one remote sensing imagewithin a predetermined time slot; obtaining a user-generated content,UGC, item, the UGC item having an associated location within thegeographical area and having been captured at a user-determined timewithin the time slot; and merging the UGC item and the remote sensingimage to form a hybrid image.
 2. The method of claim 1, wherein the UGCitem includes at least one of user generated location information, auser generated still image, a user generated video image; a usergenerated audio and wearable sensor data.
 3. The method of claim 2,wherein the UGC item is a selfie image.
 4. The method of claim 1, themethod further including obtaining ambient audio data, and wherein theUGC item further includes the ambient audio data.
 5. The method of claim1, the method further including obtaining at least one additional remotesensing image.
 6. The method of claim 5, wherein the at least oneadditional remote sensing image is obtained from a further image capturedevice.
 7. The method of claim 6, wherein said further image capturedevice is hosted by a second platform different from the satelliteplatform.
 8. The method of claim 7, wherein the second platform is oneof: a second satellite platform; a UAV platform; and a manned aircraftplatform.
 9. The method of claim 1, wherein merging comprises:processing the remote sensing image into a map projection, to compensatefor viewing angle of satellite; converting the location associated withthe UGC item into a pixel location within the processed remote sensingimage; and imposing a visible indicator on the processed remote sensingimage at the pixel location.
 10. The method of claim 1, the methodfurther including constructing a compound multimedia item by composingadditional content with the hybrid image.
 11. The method of claim 10,further comprising obtaining an historical remote sensing image of thegeographical area, wherein the additional content includes thehistorical remote sensing image.
 12. The method of claim 10, wherein theadditional content includes at least one of: metadata; additionalpre-generated video content; additional pre-generated audio content;prepared text content; prepared graphical content; visual effects; anduser generated content shared by other users.
 13. A portal device havingat least one processor and at least one communication unit, wherein theat least one communication unit is configured to receive at least oneremote sensing image of a predetermined geographical area from an imagecapture device carried on a satellite platform, the image capture devicehaving been tasked to capture the at least one remote sensing imagewithin a predetermined time slot; and to obtain a user-generatedcontent, UGC, item, the UGC item having an associated location withinthe geographical area and having been captured at a user-determined timewithin the time slot; and wherein the at least one processor isconfigured to merge the UGC item and the remote sensing image to form ahybrid image.
 14. A machine-readable medium carrying a set ofinstructions that, when executed by at least one processor, causes theat least one processor to carry out the method of claim
 1. 15. Acomputer implemented method in a mobile device comprising: causing acontent capture device to capture a user generated content, UGC, item ata user-determined time; causing a location unit to locate the mobiledevice and to associate the resulting location with the captured UGCitem, determining whether the user-determined time is within apredetermined time slot and whether the location of the mobile devicewithin a predetermined geographical area; and if the user-determinedtime is within the predetermined time slot and the location of themobile device is within the predetermined geographical area,communicating with a merging service to trigger the generation of ahybrid image that merges the user image and a remote sensing image, theremote sensing image having been captured by an image capture devicecarried on a satellite platform within the predetermined time slot andbeing of the predetermined geographical area.
 16. The method of claim15, wherein the location is obtained using a global satellitepositioning technique.
 17. The method of claim 15, wherein the locationis obtained using cell ID.
 18. A mobile device having: a content capturedevice; a location unit; a memory; and at least one processor; whereinthe memory stores instructions which when executed in the processor:cause the content capture device to capture a user generated content,UGC, item at a user-determined time; cause the location unit to locatethe mobile device and to associate the resulting location with thecaptured UGC item, determine whether the user-determined time is withina predetermined time slot and whether the location of the mobile devicewithin a predetermined geographical area; and if the user-determinedtime is within the predetermined time slot and the location of themobile device is within the predetermined geographical area, communicatewith a merging service to trigger the generation of a hybrid image thatmerges the user image and a remote sensing image, the remote sensingimage having been captured by an image capture device carried on asatellite platform within the predetermined time slot and being of thepredetermined geographical area.
 19. A machine-readable medium carryinga set of instructions that, when executed by at least one processor,causes the at least one processor to carry out the method of claim 15.